Understanding belief bias by measuring prior beliefs for a Bayesian model of syllogistic reasoning

The phenomenon of belief bias in syllogistic reasoning occurs when the a priori believability of a conclusion influences the intuitive acceptability of that conclusion. Prior beliefs about the world can be formalized into a probabilistic generative model of situations. Tessler and Goodman (2014) proposed that this very idea can account for the range of acceptabilities of conclusions from categorical syllogisms with abstract content. Here, I generalize their model to accommodate syllogistic reasoning data where content effects are observed. I collect data about the prior plausibility of various properties co-occurring, and use this data to predict syllogistic reasoning behavior in a separate experiment. I compare models with different types of assumptions concerning the prior and discuss open questions for this approach. Your logic-chopping friend is in a room with a number of lamps and lightbulbs; you are in a different room and cannot see what she sees. She gives you the following logic puzzle: All of the lightbulbs that are hot are bright. Some of the lightbulbs that are bright are not on. Are some of the hot lightbulbs not on? Are any of the hot ones on? Prior beliefs about the world guide our actions, thoughts, and reasoning in new situations. It can be helpful, for example, to know how fast a particular kind of animal can run, if you are also thinking about if that animal can eat you. Similarly, humans can use prior beliefs in a domain (e.g. life expectancies) to reason accurately about everyday contexts (e.g. guessing how long someone will live; Griffiths & Tenenbaum, 2006). Finally, it has been argued that prior beliefs influence the very meaning of words (Goodman & Lassiter, 2015). It is odd then that so little formal theory has gone into understanding prior beliefs in classic reasoning tasks (but, cf. Klauer, Musch, & Naumer, 2000; Dube, Rotello, & Heit, 2010). Bayesian approaches to cognitive science have a natural way of accounting for prior beliefs in reasoning. Tessler and Goodman (2014) described a generative model of argument strength that uses a truth-functional semantics applied to idealized situations composed of objects with properties. This model accounted for much of the variability in Chater and Oaksford (1999)’s meta-analysis data of categorical syllogistic reasoning. That work further explored syllogistic reasoning by incorporating Gricean principles, formalized in the Rational Speech-Act (RSA) theory of language understanding (Frank & Goodman, 2012; Goodman & Stuhlmüller, 2013). This pragmatic component was important in capturing important qualitative phenomena in syllogistic reasoning (e.g. the relative preference for the all X are Y conclusion over the some X are Y conclusion when both are logically valid). This work was done with respect to meta-analysis data that differed largely in the materials used, and for which prior beliefs about the materials were not expected to have a substantial effect. However, it is known that prior expectations about the categories and properties at stake in a syllogism influence the acceptability of a conclusion (J. S. Evans, Handley, & Pollard, 1983; Cherubini, Garnham, Oakhill, & Morley, 1998; J. S. Evans, Handley, & Harper, 2001). Here, I generalize Tessler and Goodman (2014)’s model of argument strength to capture qualitative phenomena associated with belief bias. This is done by empirically measuring prior beliefs about real-world content, deriving model predictions based on those beliefs, and testing the probabilisitic model of argument strength against behavioral data obtained in a separate experiment of syllogistic reasoning. A secondary, primarily methodological concern is about the granularity of information needed to capture these syllogistic reasoning phenomena. To foreshadow the results, empirically measured priors (Expt. 1) coupled with a Bayesian model of argument strength accounts for much of the syllogistic reasoning data (Expt. 2), including qualitative effects of content. The predictions of the model, which has no parameters, are as good as those of a model with a prior parametrized by 12 variables. The most likely values (conditioned on the data of Expt. 2) of these 12 variables correspond roughly with the marginal distributions of the priors elicited in Expt. 1. This interesting correspondence suggests the syllogistic reasoning task is too coarse-grained to disambiguate models of reasoning that rely on correlated properties from models where independence is assumed. 1 Bayesian argument strength in syllogistic reasoning A formal account of gradience in syllogistic reasoning was presented by Tessler and Goodman (2014). The computational model is a Bayesian model; as such, it is important to understand the implications of the prior for syllogistic reasoning. I review the model, highlighting along the way how I generalize the model to consider content effects.